The relative change of treatment effect after adjustment for small study effects is investigated in this document. Three adjustment methods are applied: Trim and fill, copas selection model and the regression model. The estimates of this methods are compared to fixed effects and random effects meta-analysis estimates.
The relative change of treatment effect after adjustment is defined as:
\(1 - |{\frac{\theta_{C}}{\theta_{M}}}|\)
were \(\theta_{M}\) is the estimate of the meta-analysis and \(\theta_{C}\) is the estimate of the adjustment method and \(||\) denotes the absolute value.
The number can be interpreted as the fraction by which the original treatment effect is reduced after adjustment. If the number is positive, the treatment estimate has been reduced by the adjustment method. If the relative change is negative, then the adjusted treatment effect estimate is larger than the estimate of the meta-analysis
First, the number of treatment effects that are smaller or larger than the treatment effects are counted:
## # A tibble: 1 x 2
## biased larger.meta.effects
## <dbl> <int>
## 1 122 1018
## # A tibble: 1 x 2
## biased smaller.meta.effects
## <dbl> <int>
## 1 32 290
## # A tibble: 1 x 2
## biased smaller.meta.effects.random
## <dbl> <int>
## 1 35 394
The empirical distribution of relative effect changes are shown in the following plots. Different ranges and different plots are shown for random and fixed effects meta-analysis and continuous and binary outcomes. Additionally, the fraction of meta-analyses with significant peters publication bias test (binary outcomes) and eggers test (continuous outcomes) is color-coded within the histogramms.
The absolute values of the treatment effect estimates can be plotted against each other. This is shown, again for different meta-analysis methods.
metac.cont %>% mutate(xx = 1 - abs(est.trimfill.fixef.cont/est.fixef.cont)) %>% filter(xx > 0) %>% ungroup() %>%
summarise(biased = sum(egger.test), larger.meta.effects = n())
## biased larger.meta.effects
## 1 34 163
metac.cont %>% mutate(xx = 1 - abs(est.trimfill.fixef.cont/est.fixef.cont)) %>% filter(xx < 0) %>% ungroup() %>%
summarise(biased = sum(egger.test), smaller.meta.effects = n())
## biased smaller.meta.effects
## 1 7 57
metac.cont %>% mutate(xx = 1 - abs(est.trimfill.fixef.cont/est.ranef.cont)) %>% filter(xx < 0) %>% ungroup() %>%
summarise(biased = sum(egger.test), smaller.meta.effects.random = n())
## biased smaller.meta.effects.random
## 1 6 78
The absolute values of the treatment effect estimates can be plotted against each other. This is shown, again for different meta-analysis methods.
The structure of the example of trim-and-fill is maintained:
## # A tibble: 1 x 2
## biased larger.meta.effects
## <dbl> <int>
## 1 96 379
## # A tibble: 1 x 2
## biased smaller.meta.effects
## <dbl> <int>
## 1 26 101
## # A tibble: 1 x 2
## biased smaller.meta.effects.random
## <dbl> <int>
## 1 22 108
## biased larger.meta.effects
## 1 26 70
## biased smaller.meta.effects
## 1 11 33
## biased smaller.meta.effects.random
## 1 9 24
## # A tibble: 1 x 2
## biased larger.meta.effects
## <dbl> <int>
## 1 105 899
## # A tibble: 1 x 2
## biased smaller.meta.effects
## <dbl> <int>
## 1 49 409
## # A tibble: 1 x 2
## biased smaller.meta.effects.random
## <dbl> <int>
## 1 42 407
## biased larger.meta.effects
## 1 27 191
## biased smaller.meta.effects
## 1 14 103
## biased smaller.meta.effects.random
## 1 8 83